Papers with Multi-modal Concept Alignment Pre-training
Multi-modal Concept Alignment Pre-training for Generative Medical Visual Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Medical Visual Question Answering (Med-VQA) aims to provide accurate answers to questions regarding medical images, a task particularly challenging for open-ended questions. |
| Approach: | They propose a multi-modal concept alignment pre-training approach for generative Med-VQA that leverages a knowledge graph sourced from medical image-caption datasets and the Unified Medical Language System. |
| Outcome: | The proposed approach significantly outperforms existing methods on a set of benchmark datasets and shows high efficiency and knowledge-image alignment capability. |